Systems and methods for migrating components in a hierarchical storage network

ABSTRACT

System and methods for selectively or automatically migrating resources between storage operation cells are provided. In accordance with one aspect of the invention, a management component within the storage operation system may monitor system operation and migrate components from storage operation cell to another to facilitate failover recovery, promote load balancing within the system and improve overall system performance as further described herein. Another aspect of the invention may involve performing certain predictive analyses on system operation to reveal trends and tendencies within the system. Such information may be used as the basis for potentially migrating components from one storage operation cell to another to improve system performance and reduce or eliminate resource exhaustion or congestion conditions.

INCORPORATION BY REFERENCE TO ANY PRIORITY APPLICATIONS

Any and all applications for which a foreign or domestic priority claimis identified in the Application Data Sheet as filed with the presentapplication are hereby incorporated by reference under 37 CFR 1.57.

RELATED APPLICATIONS

This application is related to the following patents and applications,each of which is hereby incorporated herein by reference in itsentirety:

Application Ser. No. 09/354,058, titled HIERARCHICAL BACKUP ANDRETRIEVAL SYSTEM, filed Jul. 15, 1999, now U.S. Pat. No. 7,395,282,issued Jan. 25, 2011;

U.S. Pat. No. 6,418,478, titled PIPELINED HIGH SPEED DATA TRANSFERMECHANISM, issued Jul. 9, 2002;

Application Ser. No. 10/818,749, titled SYSTEM AND METHOD FORDYNAMICALLY PERFORMING STORAGE OPERATIONS IN A COMPUTER NETWORK, filedApr. 5, 2004, now U.S. Pat. No. 7,246,207, issued Jul. 17, 2007;

Application Ser. No. 10/877,831, titled HIERARCHICAL SYSTEM AND METHODFOR PERFORMING STORAGE OPERATIONS IN A COMPUTER NETWORK, filed Jun. 25,2004, now U.S. Pat. No. 7,454,569, issued Nov. 18, 2008;

Application Ser. No. 10/990,357, SYSTEM AND METHOD FOR COMBINING DATASTREAMS IN PIPELINED STORAGE OPERATIONS IN A STORAGE NETWORK, filed Nov.15, 2004, now U.S. Pat. No. 7,315,923, issued Jan. 1, 2008;

Application Ser. No. 11/120,619, titled HIERARCHICAL SYSTEMS AND METHODSFOR PROVIDING STORAGE A UNIFIED VIEW OF STORAGE INFORMATION, filed May2, 2005, now U.S. Pat. No. 7,343,453, issued Mar. 11, 2008;

Application Ser. No. 11/269,513, titled SYSTEM AND METHOD FOR MONITORINGA STORAGE NETWORK, filed Nov. 7, 2005.

Application Ser. No. 60/752,203, titled SYSTEMS AND METHODS FORCLASSIFYING AND TRANSFERRING INFORMATION IN A STORAGE NETWORK, filedDec. 19, 2005;

Application Ser. No. 60/752,198, SYSTEMS AND METHODS FOR GRANULARRESOURCE MANAGEMENT IN A STORAGE NETWORK, filed Dec. 19, 2005;

Application Ser. No. 11/313,224, titled SYSTEMS AND METHODS FORPERFORMING MULTI-PATH STORAGE OPERATIONS filed Dec. 19, 2005;

Application Ser. No. 60/752,202, SYSTEMS AND METHODS FOR UNIFIEDRECONSTRUCTION OF DATA IN A STORAGE NETWORK filed Dec. 19, 2005;

Application Ser. No. 60/752,201, SYSTEMS AND METHODS FOR RESYNCHRONIZINGSTORAGE OPERATIONS filed Dec. 19, 2005; and

Application Ser. No. 60/752,197, SYSTEMS AND METHODS FOR HIERARCHICALCLIENT GROUP MANAGEMENT filed Dec. 19, 2005.

BACKGROUND OF THE INVENTION Field of the Invention

The invention disclosed herein relates generally to performing storageoperations on electronic data in a computer network. More particularly,the present invention relates to migrating resources used in performingstorage operations from one storage operation cell to another.

Storage management systems have evolved over time into complex entitieswith many components including hardware and software modules designed toperform a variety of different storage operations on electronic data.Current storage management systems employ a number of different methodsto perform storage operations on electronic data. For example, data canbe stored in primary storage as a primary copy or in secondary storageas various types of secondary copies including, as a backup copy, asnapshot copy, a hierarchical storage management copy (“HSM”), as anarchive copy, and as other types of copies.

A primary copy of data is generally a production copy or other “live”version of the data which is used by a software application and istypically in the native format of that application. Primary copy datamay be maintained in a local memory or other high-speed storage devicethat allows for relatively fast data access. Such primary copy data istypically retained for a period of time (e.g., a number of seconds,minutes, hours or days) before some or all of the data is stored as oneor more secondary copies, for example, to prevent loss of data in theevent a problem occurs with the data stored in primary storage.

Secondary copies may include point-in-time data and may be intended forlong-term retention (e.g., weeks, months or years depending on retentioncriteria, for example as specified in a storage policy as furtherdescribed herein) before some or all of the data is moved to otherstorage or discarded. Secondary copies may be indexed so users canbrowse and restore the data at another point in time. After certainprimary copy data is copied to secondary storage, a pointer or otherlocation indicia such as a stub may be placed in the primary copy toindicate the current location of that data.

One type of secondary copy is a backup copy. A backup copy is generallya point-in-time copy of the primary copy data stored in a backup formatas opposed to in native application format. For example, a backup copymay be stored in a backup format that is optimized for compression andefficient long-term storage. Backup copies generally have relativelylong retention periods and may be stored on media with slower retrievaltimes than other types of secondary copies and media. In some cases,backup copies may be stored at an offsite location.

Another form of secondary copy is a snapshot copy. From an end-userviewpoint, a snapshot may be thought of as a representation or image ofthe primary copy data at a given point in time. A snapshot generallycreates a bit map or block level representation of a primary copy volumeat a particular moment in time. Users typically gain a read-only accessto the record of files and directories of the snapshot. By electing torestore primary copy data from a snapshot taken at a given point intime, users may also return the current file system to the prior stateof the file system that existed when the snapshot was taken.

A snapshot may be created instantly, using a minimum of file space, butmay still function as a conventional file system backup. A snapshot maynot actually create another physical copy of all the data, but maysimply create pointers that are mapped to specific blocks of data takenat the point in time of the snapshot.

In some conventional systems, once a snapshot has been taken, theoriginal blocks in use at the time at snapshot are preserved in a cachesuch that only subsequent changes to the file system would overwritethem. Therefore, the initial snapshot may use only a small amount ofdisk space needed to record a mapping or other data structurerepresenting or otherwise tracking the blocks that correspond to thecurrent state of the volume (e.g., a bit map). Additional disk space isusually only required when files are actually modified later.

Furthermore, when files are modified, typically only the pointers whichmap to blocks are copied, not the blocks themselves. In some systems,for example in the case of copy-on-write snapshots, when a block changesin primary storage, the block is copied to secondary storage before theblock is overwritten in primary storage and the snapshot mapping of filesystem data is updated to reflect the changed block(s) at thatparticular point in time.

A Hierarchical Storage Management (“HSM”) copy is generally a copy ofthe primary copy data, but typically includes only a subset of theprimary copy data that meets a certain criteria and is usually stored ina format other than the native application format. For example, an HSMcopy might include only that data from the primary copy that is largerthan a given size threshold or older than a given age threshold and thatis stored in a backup format. Often, HSM data is removed from theprimary copy, and a stub is stored in the primary copy to indicate itsnew location. When a user subsequently requests access to the HSM datathat has been removed or migrated, systems then use the stub to locatethe data and often make recovery of the data appear transparent, eventhough the HSM data may be stored at a location different from theremaining primary copy data.

An archive copy is generally similar to an HSM copy, however, the datasatisfying criteria for removal from the primary copy is generallycompletely removed with no stub left in the primary copy to indicate thenew location (i.e., where it has been moved to). Archive copies of dataare generally stored in a backup format or other non-native applicationformat. In addition, archive copies are generally retained for very longperiods of time (e.g., years) and in some cases are never deleted. Sucharchive copies may be made and kept for extended periods in order tomeet compliance regulations or for other permanent storage applications.

In some systems, data may be moved over its lifetime moves fromexpensive quick access storage, to less expensive slower access storage.This process of moving data through these various tiers of storage issometimes referred to as information lifecycle management (“ILM”). Thisis the process by which data is “aged” from more expensive forms ofsecondary storage with faster access/restore times down through lessexpensive secondary storage with slower access/restore times, forexample, as the data becomes less important or mission critical overtime.

Examples of various types of data and copies of data are furtherdescribed in the above-referenced related applications that are herebyincorporated by reference in their entirety. One example of a systemthat performs storage operations on electronic data that produce suchcopies is the QINETIX storage management system by COMMVAULT Systems ofOceanport, N.J.

The QINETIX system provides a modular storage management architecturethat may include, among other things, storage manager components, clientor data agent components, and media agent components as furtherdescribed in U.S. patent application Ser. No. 10/818,749 which is herebyincorporated herein by reference in its entirety.

The QINETIX system also provides for hierarchically configuring storageoperation cells to perform certain other types of storage operationsincluding snapshot copies, HSM copies, archive copies, and certain othertypes of copies of electronic data, as further described in U.S. patentapplication Ser. No. 11/120,619 which is hereby incorporated byreference in its entirety.

The QINETIX system, as further disclosed in U.S. patent application Ser.No. 11/120,619, also provides for meaningful integration between andamong HSM systems and Storage Resource Management (“SRM”) systems. SRMsystems are typically concerned with monitoring the health, status, andother information associated with primary copies of data (e.g., live orproduction line copies). For example, an SRM application may monitor thecapacity of a volume storing a primary copy of data, and the rate ofdata transfer relative to the available storage of that volume. Incontrast, HSM systems are typically concerned with data transport tosecondary storage, such as the parameters and particulars associatedwith the storage and migration of secondary copy data as opposed tomonitoring of information associated with primary copy data. HSM systemsare thus generally directed to secondary copies and other archival datavolumes. For example, an HSM system may be directed to migrating orotherwise transferring data to a secondary copy.

Transferring data to any one of the aforementioned secondary copy typesmay be performed using a storage management system, using one or morecopy devices, components or other data transfer resources. Depending onstorage requirements, secondary copy operations or other storageoperations may be performed at various times which may result in anundesirably high number of storage operations being performed at aparticular time using the same or similar resources. Meanwhile, othersecondary copy resources may be underutilized.

In some instances, it may be desirable to migrate or reassign storageresources from one storage operation system, or subset of a system, suchas a cell, to another to provide additional routing options or to helpalleviate any bottlenecks or congestion that may occur based onestablished commitments or system configuration.

There are also situations where storage resources may be physicallymoved, and certain storage management components should be migrated tonew places in the system to continue to support the moved storageresources.

Moreover, in certain situations, component migration, reallocation ormovement, may be desired or necessary, such as in a disaster recovery(DR) situation where data and/or resources are temporarily orpermanently moved to another storage operation cell.

SUMMARY OF THE INVENTION

Aspects of the present invention are directed towards systems andmethods for selectively or automatically migrating storage systemresources between various storage operation cells. In accordance withone aspect of the invention, a management component within the storageoperation system may monitor system operation and migrate componentsfrom storage operation cell to another to facilitate failover recovery,promote load balancing within the system and improve overall systemperformance as further described herein. Another aspect of the inventionmay involve performing certain predictive analyses on system operationto identify trends and/or tendencies within the system. Such informationmay be used as the basis for potentially migrating components from onestorage operation cell to another to improve system performance andreduce or eliminate resource exhaustion or congestion conditions.

In one illustrative embodiment of the invention, a method forautomatically transferring a resource from a first storage operationcell to a second storage operation cell is provided. The method includesmonitoring a first operational characteristic of a first resource in thefirst storage operation cell wherein the first operationalcharacteristic has an associated first threshold; identifying a secondstorage operation cell including a second resource having a secondoperational characteristic that does not exceed a second threshold;determining whether the first resource may be transferred to the secondstorage operation cell such that the transfer is capable of causing thefirst operational characteristic to remain within the first threshold;and automatically transferring the first resource to the second storageoperation cell if the first operational characteristic is near orexceeds the first threshold.

In accordance with another embodiment of the present invention, a systemfor optimizing performance of storage operations in a storage systemincluding a plurality of storage operation cells and resources isprovided. The system may include a storage manager that monitors anoperational characteristic of a plurality of storage operation cells inthe storage system; wherein the storage manager identifies a group firstresources in a first storage operation cell and identifies a pluralityof second storage operation cells to which the group of first resourcesmay be transferred; and wherein the storage manager determines apotential effect that a transfer of the group of first resources to theplurality of second storage operation cells would have on theperformance storage operations, and if the determined potential effectof a transfer of the group of first resources improves the performanceof storage operations, the storage manager transfers the group of firstresources to the plurality of second storage operation cells.

BRIEF DESCRIPTION OF THE DRAWINGS

Aspects of the invention are illustrated in the figures of theaccompanying drawings which are meant to be exemplary and not limiting,in which like references are intended to refer to like or correspondingparts, and in which:

FIG. 1 is a block diagram of a storage operation cell in a system toperform storage operations on electronic data in a computer networkaccording to an embodiment of the invention;

FIG. 2 is a block diagram of a hierarchically organized group of storageoperation cells in a system to perform storage operations on electronicdata in a computer network, according to an embodiment of the invention;

FIG. 3 is a block diagram of a hierarchically organized group of storageoperation cells in a system to perform resource migration betweenstorage operation cells according to an embodiment of the invention;

FIG. 4 schematically depicts migration of various clients and mediaagents between storage operation cells in the hierarchically organizednetwork of FIG. 3, according to an embodiment of the invention;

FIG. 5 is a flow diagram of a method for migrating one or more resourcesfrom one storage operation cell to another, according to an embodimentof the invention;

FIG. 6 is a flow diagram of a method for dynamic resource allocation,according to an embodiment of the invention; and

FIG. 7 is a flow diagram of a method for resource allocation based onpredictive analysis, according to an embodiment of the invention;

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

Systems and methods for migrating one or more components of a storageoperation cell in a storage network, and for predicting certainrequirements of storage operation cell components within the network areprovided. In various illustrative embodiments, the present invention,among other things, provides for migrating or “reallocating” resourcesor components from one storage operation cell to another storageoperation cell such that the migrated resources from the “source”storage operation cell become part of and operate with resources in thedestination or “target” storage operation cell. Such migration typicallyinvolves logical reassignment of certain resources from one storageoperation cell to another (i.e., no actual physical movement), but, insome embodiments, may include a physical movement of resources orcomponents. For example, a change of a logical association orreassignment of resources or components may include modifyingassociations or groups as defined in certain management softwarecomponents which may involve changing a pointer, address or otherreferential element in a storage management system index or database andmay indicate a new or changed network pathway to a migrated resource.Physical movement however, may involve actual physical movement of someor all components from one location to another (which also, in someembodiments, may involve logical reassignment or update in managementsoftware, e.g., new cabling, routing, etc.). Migrated resources mayinclude clients, media agents, storage managers, storage devices and/orcertain associated storage media, clients, transmission resources suchas pathways, switches, routers or other resources or components.

The migration of resources between storage operation cells may occur invarious situations, and may be temporary or permanent. Some examples ofscenarios in which resource migration maybe desirable include: failureof one or more storage operation cell resources (e.g., if one or morestorage operation cell resources fails, a resource may be identified inanother storage operation cell to replace the failed storage operationcell), merging storage operation cell resources (e.g., due to certainconditions within an enterprise or environment, storage operation cellsmay be merged to improve system performance or otherwise consolidatesystem resources, or to be substituted for other storage operation cellswhich may be removed), assisting in alleviating or reducing networkcongestion (e.g., one or more data pathways may be overburdened with ahigh volume of data transfers), changes or efficiency in using networkpathways (e.g., one network pathway may be taken offline, or a networkpathway may have comparatively faster/slower data transfercharacteristics), load balancing (e.g., one or more media agents,storage devices or other component may be migrated to handle a highvolume of data transfer streams or to more evenly distribute workload),addition or removal of storage operation cell resources (e.g., if a newstorage operation cell resource may be migrated to a storage operationcell having the greatest need for such new resource, or if a resource isremoved, another resource may be migrated to replace the removedresource), or other situations to facilitate certain system goals suchresource optimization or data security which may arise in a storagemanagement system.

Thus, in general, to optimize performance of storage operations in astorage management system, storage operation cell resources may bemigrated to more evenly distribute workload, promote failover responseand system robustness, take advantage of particular storagecharacteristics of a particular resource for a storage operation, merge,reorganize or otherwise reconfigure system resources, information, ordata, improve or maintain network accessibility and data protection, orimprove any other performance characteristics as contemplated by variousaspects of the invention.

As further described herein, resource migration may be performed underuser supervision (e.g., by commands or scripts input by a user via aGUI); or automatically (i.e., without user supervision; e.g., bymigration or management components throughout a network or by storagemanagers) based on user set parameters, a threshold, a storage policy,storage preference or other preference as further described herein; orthrough any combination of user commands and automatically executingcomponents. Storage operation cell migration operations may be initiatedautomatically as a suggested, corrective, or preemptive action withrespect to enhancing system performance (e.g., to improve performance invarious storage operation scenarios as described herein); or for otherreasons (e.g., consolidation or reconfiguration, etc.).

One exemplary storage management system which may provide storageoperation cell resource migration is described with reference to FIGS. 1through 7, although it will be understood other storage managementsystem architectures known in the art may also use the principles of theinventions described herein. FIG. 1 illustrates a block diagram of anillustrative storage operation cell 50 that may perform storageoperations on electronic data in a computer network in accordance withan embodiment of the present invention. As shown, storage operation cell50 may generally include a storage manager 100, a data agent 95, a mediaagent 105, a storage device 115, and, in some embodiments, may includecertain other components such as a client 85, a data or informationstore 90, databases 110 and 111, jobs agent 120, an interface module125, and a management agent 130. Such system and elements thereof areexemplary of a modular storage operation system such as the COMMVAULTQINETIX system, and also the COMMVAULT GALAXY storage operation system,available from COMMVAULT Systems, Inc. of Oceanport, N.J., and furtherdescribed in U.S. Pat. No. 7,035,880, which is incorporated herein byreference in its entirety.

A storage operation cell, such as cell 50, may generally includecombinations of hardware and software components associated withperforming storage operations on electronic data. Exemplary storageoperation cells according to embodiments of the invention may include,as further described herein, CommCells as embodied in the QNet storagemanagement system and the QINETIX storage management system by COMMVAULTSystems of Oceanport, N.J.

Additional storage operations performed by storage operation cells mayinclude creating, storing, retrieving, and migrating primary data copiesand secondary data copies (which may include, for example, snapshotcopies, backup copies, HSM copies, archive copies, and other types ofcopies of electronic data). In some embodiments, storage operation cellsmay also provide one or more integrated management consoles for users orsystem processes to interface with in order to perform certain storageoperations on electronic data as further described herein. Suchintegrated management consoles may be displayed at a central controlfacility or several similar consoles distributed throughout multiplenetwork locations to provide global or geographically specific networkdata storage information.

Storage operations may be performed according to a storage policy. Astorage policy is generally a data structure or other information sourcethat includes a set of preferences and other storage criteria forperforming a storage operation. The preferences and storage criteria mayinclude, but are not limited to, a storage location, relationshipsbetween system components, network pathway to utilize, retentionpolicies, data characteristics, compression or encryption requirements,preferred system components to utilize in a storage operation, and othercriteria relating to a storage operation. Thus, a storage policy mayindicate that certain data is to be stored in a specific storage device,retained for a specified period of time before being aged to anothertier of secondary storage, copied to secondary storage using a specifiednumber of streams, etc. A storage policy may be stored to a storagemanager database 111, to archive media as metadata for use in restoreoperations or other storage operations, or to other locations orcomponents of the system.

A schedule policy may specify when to perform storage operations and howoften and may also specify performing certain storage operations onsub-clients of data and how to treat those sub-clients. A sub-client mayrepresent static or dynamic associations of portions of data of a volumethat are mutually exclusive. Thus, a portion of data may be given alabel, and the association may be stored as a static entity in an index,database or other storage location used by the system. Sub-clients mayalso be used as an effective administrative scheme of organizing dataaccording to data type, department within the enterprise, storagepreferences, etc. For example, an administrator may find it preferableto separate e-mail data from financial data using two differentsub-clients having different storage preferences, retention criteria,etc.

Storage operation cells may contain not only physical devices, but alsomay represent logical concepts, organizations, and hierarchies. Forexample, a first storage operation cell 50 may be configured to performHSM operations, such as data backup or other types of data migration,and may include a variety of physical components including a storagemanager 100 (or management agent 130), a media agent 105, a clientcomponent 85, and other components as described herein. A second storageoperation cell may contain the same or similar physical components,however, it may be configured to perform other storage operations, suchas monitoring a primary data copy or performing other known storageoperations.

Thus, as can be appreciated from the above, although the first andsecond storage operation cells are logically distinct entities and maybe configured to perform different management functions (e.g., HSM andSRM respectively), each storage operation cell may contain the same orsimilar physical devices. Alternatively, in other embodiments, differentstorage operation cells may contain some or none of the same physicaldevices. For example, a storage operation cell 50 configured to performSRM tasks may contain a media agent 105, client 85, or other networkdevice connected to a primary storage volume, while a storage operationcell 50 configured to perform HSM tasks may instead include a mediaagent 105, client 85, or other network device connected to a secondarystorage volume and not contain the elements or components associatedwith and including the primary storage volume. These two storageoperation cells, however, may each include a different storage manager100 that coordinates storage operations via the same media agents 105and storage devices 115. This “overlapping” configuration allows storageresources to be accessed by more than one storage manager 100 such thatmultiple paths exist to each storage device 115 facilitating failover,load balancing, and promoting robust data access via alternative routes.

Alternatively, in some embodiments, the same storage manager 100 maycontrol two or more storage operation cells 50 (whether or not eachstorage operation cell 50 has its own dedicated storage manager 100).Moreover, in certain embodiments, the extent or type of overlap may beuser-defined (e.g., through a control console (not shown)) or may beautomatically configured to optimize data storage and/or retrieval.

Data agent 95 may be a software module or part of a software module thatis generally responsible for archiving, migrating, identifying, andrecovering data from client computer 85 stored in an information store90 or other memory location. Each client computer 85 may have at leastone data agent 95 and the system can support multiple client computers85. In some embodiments, data agents 95 may be distributed betweenclient 85 and storage manager 100 (and any other intermediate components(not shown)) or may be deployed from a remote location or its functionsapproximated by a remote process that performs some or all of thefunctions of data agent 95.

Embodiments of the present invention may employ multiple data agents 95each of which may perform a storage operation, e.g., backup, migrate,and recover data associated with a different application. For example,different individual data agents 95 may be designed to handle MICROSOFTEXCHANGE® data, Lotus Notes data, MICROSOFT WINDOWS® 2000 file systemdata, MICROSOFT Active Directory Objects data, and other types of dataknown in the art. Other embodiments may employ one or more generic dataagents 95 that can handle and process multiple data types rather thanusing the specialized data agents described above.

If a client computer 85 has two or more types of data, one data agent 95may be required for each data type to archive, migrate, and restore theclient computer 85 data. For example, to backup, migrate, and restoreall of the data on a MICROSOFT EXCHANGE® 2000 server, the clientcomputer 85 may use one MICROSOFT EXCHANGE® 2000 Mailbox data agent 95to backup the EXCHANGE 2000 mailboxes, one MICROSOFT EXCHANGE® 2000Database data agent 95 to backup the EXCHANGE 2000 databases, oneMICROSOFT EXCHANGE® 2000 Public Folder data agent 95 to backup theEXCHANGE 2000 Public Folders, and one MICROSOFT WINDOWS® 2000 FileSystem data agent 95 to backup the client computer's 85 file system.These data agents 95 would be treated as four separate data agents 95 bythe system even though they reside on the same client computer 85.

Alternatively, other embodiments may use one or more generic data agents95, each of which may be capable of handling two or more data types. Forexample, one generic data agent 95 may be used to back up, migrate andrestore MICROSOFT EXCHANGE® 2000 Mailbox data and MICROSOFT EXCHANGE®2000 Database data while another generic data agent may handle MICROSOFTEXCHANGE® 2000 Public Folder data and MICROSOFT WINDOWS® 2000 FileSystem data, etc.

Generally speaking, storage manager 100 may be a software module orother application that coordinates and controls storage operationsperformed by storage operation cell 50. Storage manager 100 maycommunicate with some or all elements of the storage operation cell 50including client computers 85, data agents 95, media agents 105, andstorage devices 115, to initiate and manage system backups, migrations,and data recovery.

Storage manager 100 may include a jobs agent 120 that monitors thestatus of some or all storage operations previously performed, currentlybeing performed, or scheduled to be performed by storage operation cell50. Jobs agent 120 may be communicatively coupled with an interfaceagent 125 (typically a software module or application). Interface agent125 may include information processing and display software, such as agraphical user interface (“GUI”), an application program interface(“API”), or other interactive interface through which users and systemprocesses can retrieve information about the status of storageoperations. Through interface 125, users may optionally issueinstructions to various storage operation cells 50 regarding performanceof the storage operations as described and contemplated by the presentinvention. For example, a user may modify a schedule concerning thenumber of pending snapshot copies (or other types of copies) scheduledas needed to suit particular needs or requirements. As another example,a user may employ the GUI to view the status of pending storageoperations in some or all of the storage operation cells in a givennetwork; or to monitor the status of certain components in a particularstorage operation cell (e.g., the amount of storage capacity left in aparticular storage device). As a further example, interface 125 maydisplay the cost metrics associated with a particular type of datastorage and may allow a user to determine overall and target costmetrics associated with a particular data type, or certain storageoperation cell 50 or other storage operation as predefined oruser-defined (discussed in more detail below).

Storage manager 100 may also include a management agent 130 that istypically implemented as a software module or application program. Ingeneral, management agent 130 provides an interface that allows variousmanagement components 100 in other storage operation cells 50 tocommunicate with one another. For example, assume a certain networkconfiguration includes multiple storage operation cells 50 adjacent toone another or otherwise logically related in a WAN or LAN configuration(not shown). With this arrangement, each storage operation cell 50 maybe connected to the other through each respective interface module 125.This allows each storage operation cell 50 to send and receive certainpertinent information from other storage operation cells 50, includingstatus information, routing information, information regarding capacityand utilization, etc. These communication paths may also be used toconvey information and instructions regarding storage operations.

For example, a management agent 130 in first storage operation cell 50may communicate with a management agent 130 in a second storageoperation cell 50 regarding the status of storage operations in thesecond storage operation cell. Another illustrative example includes thecase where a management agent 130 in first storage operation cell 50communicates with a management agent 130 in a second storage operationcell to control the storage manager 100 (and other components) of thesecond storage operation cell via the management agent 130 contained inthe storage manager 100.

Another illustrative example is the case where management agent 130 inthe first storage operation cell 50 communicates directly with andcontrols the components in the second storage operation cell 50 andbypasses the storage manager 100 in the second storage operation cell.If desired, storage operation cells 50 can also be organizedhierarchically such that hierarchically superior storage operation cellscontrol or pass information to hierarchically subordinate storageoperation cells or vice versa. More information regarding hierarchicalorganization of operation storage cells 50 will be provided below.

Storage manager 100 may also maintain an index cache, a database, orother data structure 111. The data stored in database 111 may be used toindicate logical associations between components of the system, userpreferences, management tasks, or other useful data. As furtherdescribed herein, some of this information may be stored in a mediaagent database or other local data store according to some embodiments.For example, the storage manager 100 may use data from database 111 totrack logical associations between media agents 105 and storage devices115.

Generally speaking, a media agent 105 may be implemented as softwaremodule that conveys data, as directed by a storage manager 100, betweena client computer 85 and one or more storage devices 115 such as a tapelibrary, a magnetic media storage device, an optical media storagedevice, or any other suitable storage device. In one embodiment, mediaagents 105 may be communicatively coupled with and control a storagedevice 115 associated with that particular media agent. A media agent105 may be considered to be associated with a particular storage device115 if that media agent 105 is capable of routing and storing data toparticular storage device 115.

In operation, a media agent 105 associated with a particular storagedevice 115 may instruct the storage device to use a robotic arm or otherretrieval means to load or remove a certain storage media, and tosubsequently archive, migrate, or restore data to or from that media.Media agents 105 may communicate with a storage device 115 via asuitable communications path such as a SCSI or fiber channelcommunications link. In some embodiments, the storage device 115 may becommunicatively coupled to a media agent 105 via a Storage Area Network(“SAN”).

Each media agent 105 may maintain an index cache, a database, or otherdata structure 110 which stores index data generated during backup,migration, and restore and other storage operations as described herein.For example, performing storage operations on MICROSOFT EXCHANGE® datamay generate index data. Such index data provides a media agent 105 orother external device with a fast and efficient mechanism for locatingdata stored or backed up. Thus, in some embodiments, a storage managerdatabase 111 may store data associating a client 85 with a particularmedia agent 105 or storage device 115, for example, as specified in astorage policy, while media agent database 110 may indicate wherespecifically the client 85 data is stored in storage device 115, whatspecific files were stored, and other information associated withstorage of client 85 data. In some embodiments, such index data may bestored along with the data backed up in a storage device 115, with anadditional copy of the index data written to index cache 110. The datain index cache 110 is thus readily available for use in storageoperations and other activities without having to be first retrievedfrom the storage device 115.

In some embodiments, certain components may reside and execute on thesame computer. For example, in some embodiments, a client computer 85such as a data agent 95, a media agent 105, or a storage manager 100coordinates and directs local archiving, migration, and retrievalapplication functions as further described in U.S. Pat. No. 7,035,880.This client computer 85 can function independently or together withother similar client computers 85.

FIG. 2 presents a generalized block diagram of a hierarchicallyorganized group of storage operation cells in a system to performstorage operations on electronic data in a computer network inaccordance with an embodiment of the present invention. It will beunderstood that although the storage operation cells generally depictedin FIG. 2 have different reference numbers from the storage operationcell 50 shown in FIG. 1, these storage operation cells may be configuredthe same as or similar to the storage operation cell 50 depicted in FIG.1.

As shown, the system illustrated in FIG. 2 may include a master storagemanager component 135 and various other storage operation cells. Asshown, the illustrative embodiment in FIG. 2 includes a first storageoperation cell 140, a second storage operation cell 145, a third storageoperation cell 150, a fourth storage operation cell 155, a fifth storageoperation cell 160, and an nth storage operation cell 165. However, itwill be understood this illustration is only exemplary and that fewer ormore storage operation cells may be present or interconnecteddifferently if desired.

Storage operation cells, such as the ones shown in FIG. 2 may becommunicatively coupled and hierarchically organized. For example, amaster storage manager component 135 may be associated with, communicatewith, and direct storage operations for, a first storage operation cell140, a second storage operation cell 145, a third storage operation cell150, a fourth storage operation cell 155, a fifth storage operation cell160, and an nth storage operation cell 165. In some embodiments, themaster storage manager component 135 may not be part of any particularstorage operation cell. In other embodiments (not shown), master storagemanager component 135 may itself be part of a certain storage operationcell.

Thus, in operation, master storage manager component 135 may communicatewith a management agent of the storage manager of the first storageoperation cell 140 (or directly with the other components of firststorage operation cell 140) with respect to storage operations performedin the first storage operation cell 140. For example, in someembodiments, master storage manager component 135 may instruct the firststorage operation cell 140 with certain commands regarding a desiredstorage operation such as how and when to perform particular storageoperations including the type of operation and the data on which toperform the operation.

In other embodiments, master storage manager component 135 may track thestatus of its associated storage operation cells, such as the status ofjobs, system components, system resources utilization, and other items,by communicating with manager agents (or other components) in therespective storage operation cells. Moreover, master storage managercomponent 135 may track the status of its associated storage operationcells by receiving periodic status updates from the manager agents (orother components) in the respective storage operation cells regardingjobs, system components, system resources, and other items. For example,master storage manager component 135 may use methods to monitor networkresources such as mapping network pathways and topologies to, amongother things, physically monitor storage operations and suggest, forexample, alternate routes for storing data as further described herein.

In some embodiments, master storage manager component 135 may storestatus information and other information regarding its associatedstorage operation cells and other system information in an index cache,database 137 or other data structure accessible to mater storage managercomponent 135. A presentation interface included in certain embodimentsof master storage manager component 135 may access this information andpresent it to users and system processes with information regarding thestatus of storage operations, storage operation cells, systemcomponents, and other information of the system.

As mentioned above, storage operation cells may be organizedhierarchically. With this configuration, storage operation cells mayinherit properties from hierarchically superior storage operation cellsor be controlled by other storage operation cells in the hierarchy(automatically or otherwise). Thus, in the embodiment shown in FIG. 2,storage operation cell 145 may control or is otherwise hierarchicallysuperior to storage operation cells 150, 155, 160, and 165. Similarly,storage operation cell 155 may control storage operation cells 160 and165. Alternatively, in some embodiments, storage operation cells mayinherit (or otherwise be associated with) storage policies, storagepreferences, storage metrics, or other properties or characteristicsaccording to their relative position in a hierarchy of storage operationcells.

Storage operation cells may also be organized hierarchically accordingto function, geography, architectural considerations, or other factorsuseful or desirable in performing storage operations. For example, inone embodiment, storage operation cell 140 may be directed to createsnapshot copies of primary copy data, while storage operation cell 145may be directed to create backup copies of primary copy data or otherdata.

As another example, storage operation cell 140 may represent ageographic segment of an enterprise, such as a Chicago office, andstorage operation cell 145 may represents a different geographicsegment, such as a New York office. In this example, the second storageoperation cells 145, 150, 155, 160, and 165 may represent departmentswithin the New York office. Alternatively, these storage operation cellscould be further divided by function performing various storageoperations for the New York office or load balancing storage operationsfor the New York office.

As another example, and as previously described herein, differentstorage operation cells directed to different functions may also containthe same or a subset of the same set of physical devices. Thus, onestorage operation cell in accordance with some embodiments of thepresent invention may be configured to perform SRM operations (such as aQSM cell in the COMMVAULT QINETIX system) and may contain the same,similar or a subset of the same physical devices as a cell configured toperform HSM or other types of storage operations. Each storage operationcell may, however, share the same parent or, alternatively, may belocated on different branches of a storage operation cell hierarchytree. For example, storage operation cell 140 may be directed to SRMoperations whereas storage operation cell 155 may be directed to HSMoperations. Similarly, storage operation cell 150 may be configured toperform SRM operations and storage operation cell 155 may be configuredto perform HSM operations. Those skilled in the art will recognize thata wide variety of such combinations and arrangements of storageoperation cells are possible to address a broad range of differentaspects of performing storage operations in a hierarchy of storageoperation cells.

In some embodiments, hierarchical organization of storage operationcells facilitates, among other things, system security and otherconsiderations. For example, in some embodiments, only authorized usersmay be allowed to access or control certain storage operation cells. Anetwork administrator for an enterprise may have access to many or allstorage operation cells including master storage manager 135. But anetwork administrator for the New York office, according to a previousexample, may only have access to storage operation cells 145-165, whichform the New York office storage management system.

The hierarchical organization of storage operation cells may facilitatestorage management planning and decision-making. For example, in someembodiments, data and other information related to HSM storage operationcells and SRM storage operation cells may be normalized (e.g., convertedinto a common format with normalized identifiers and other information)and combined by a master storage manager component or otherhierarchically superior (or, if desired, hierarchically subordinate)component, and a user of master storage manager component 135 may viewthe status of some or all jobs in associated storage operation cells aswell as the status of each component of the storage operation cellspresent in the system. Users may then plan and make decisions based onthis global data. For example, a user may view high-level informationregarding storage operations for the entire system, such as job status,component status, resource status (such as network pathways, etc.), andother information. This may be done from an SRM perspective to viewprimary copy information or from an HSM perspective to view secondarycopy information. A user may also drill down through menus or use othermeans to obtain more detailed information regarding a particular storageoperation cell or group of storage operation cells.

In other embodiments master storage manager component 135 may alert auser such as a system administrator when a particular resource isunavailable or congested. For example, a particular storage device mightbe full or require additional media. As another example, a masterstorage manager component may use information from an HSM storageoperation cell and an SRM storage operation cell to present indicia orotherwise alert a user or otherwise identify aspects of storageassociated with the storage management system and hierarchy of storageoperation cells.

Alternatively, a storage manager in a particular storage operation cellmay be unavailable due to hardware failure, software problems, or otherreasons. In some embodiments, master storage manager component 135 (oranother storage manager within the hierarchy of storage operation cells)may utilize the global data regarding its associated storage operationcells to suggest solutions to such problems when they occur (or act as awarning prior to occurrence). For example, master storage managercomponent 135 may alert the user that a storage device or other resourcein a particular storage operation cell is full or otherwise congested,and then suggest, based on job and data storage information contained inits database, an alternate storage device or resource to migrate to thestorage operation cell. Other types of corrective actions based on suchinformation may include suggesting an alternate data path to aparticular storage device, or dividing data to be stored among variousavailable storage devices as a load balancing measure or to otherwiseoptimize storage or retrieval time. In some embodiments, suchsuggestions or corrective actions may be performed automatically, ifdesired.

Examples of the types of information that may be monitored for anapplication such as MICROSOFT EXCHANGED include:

File Systems and File Analysis

Distribution by file types

Distribution by file sizes

Distribution by modification time/access time

Distribution by owner, etc.

Capacity and Asset Reporting

By host

By disk

By partitions

Availability Reporting

Disks

Hosts

Applications

Message Level

Distribution across mailboxes

By Message attachment type

By Message age

Data Migration report

Usage Reporting

By Storage Group

By Store

By Mailbox, etc.

In alternate embodiments, HSM and SRM components may be aware of eachother due to a common database or metabase of information, which mayinclude normalized data. Therefore, in those embodiments there is noneed for such information to pass through a master storage manager asthese components may be able to communicate directly with one another.For example, storage operation cell 150 may communicate directly withstorage operation cell 165 and vice versa. This may be accomplishedthrough a direct communications link between the two (not shown) or bypassing data through intermediate storage operation cells, such asstorage operation cells 145-160 in this case.

FIG. 3 illustrates a block diagram of a hierarchically organized groupof storage operation cells in a system to perform SRM and HSM storageoperations on electronic data in a computer network in accordance withvarious illustrative implementations for migrating resources betweenstorage operation cells in accordance with the present invention. Aswill be further understood from the following description, however,implementing storage operation cell component migration according to thepresent invention is not limited to having a particular organization ofstorage operation cells, nor to having storage operation cells with HSMand/or SRM component functionality. For instance, the storage operationcells may be organized non-hierarchically or hierarchically, and may beimplemented with or without a storage operation cell having a storagemanager that functions as a master storage manager. As a furtherexample, the network is not required to include a storage operation cellthat performs SRM operations and not HSM storage operations.

As shown, the network in FIG. 3 includes a first storage operation cell170, a second storage operation cell 173, a third storage operation cell175, and a fourth storage operation cell 180. First storage operationcell 170 includes a client 185 in communication with a primary volume190 storing data, and a storage manager component 195 in communicationwith a storage manager database 200 and with client 185. Second storageoperation cell 173 includes a client 186 in communication with a primaryvolume 191, a media agent 206 in communication with a secondary storagemedia 211 and a secondary storage media 213, clients 236-244 incommunication with respective primary volumes (not shown for clarity ofillustration), media agents 208, 210, and 212 in communication with oneor more of storage volumes 211, 213, 217, and 219, and storage manager196 in communication with each of the media agents 206, 208, 210, and212 in storage operation cell 173. Third storage operation cell 175includes clients 246 and 250 in communication with respective primaryvolumes (not shown), media agents 214, 216, and 218 in respectivecommunication with secondary storage media 221, 223, and 225, andstorage manager 260 in communication with each of the media agents instorage operation cell 175. Fourth storage operation cell includes amaster storage manager component 215 in communication with a masterstorage manager database 220.

With regard to FIG. 3, databases 220 and 200 respectively associatedwith storage managers 215 and 195 are expressly depicted, storagemanagers 196 and 260 are similarly each in communication with arespective database; however, these databases are not separatelydepicted and may be considered as being included in storage managers 196and 260, in accordance with the illustrative storage manager 100 ofFIG. 1. Additionally, in this embodiment, each depicted client device instorage operation cells 173 and 175 is in communication with the storagemanager within its storage operation cell, but connecting lines are notshown for clarity of illustration. Depicted lines connecting mediaagents with clients and with storage volumes represent possible dataflow paths according to the illustrative configuration. Thus, a givenmedia agent may be operative in storage operations (e.g., backup,archiving, restoring) with respect to any storage volume and any clientto which it is illustrated as being connected by a solid line. Depictedlines connecting a media agent to a storage manager represent controlinformation and/or metadata paths.

The first storage operation cell 170 may be configured to perform aparticular type of storage operation, such as SRM storage operations.For example, the first storage operation cell 170 may monitor andperform SRM-related calculations and operations associated with primarycopy data. Thus, the first storage operation cell 170 includes a clientcomponent 185 in communication with a primary volume 190 for storingdata. For example, client 185 may be directed to using EXCHANGE data,SQL data, Oracle data, or other types of production data used inbusiness applications (or other applications) and stored in primaryvolume 190. Storage manager component 195 may contain modules or otherlogic directed to monitoring or otherwise interacting with attributes,characteristics, metrics, and other information associated with the datastored in primary volume 190. Storage manager 195 may track and storethis and other information in storage manager database 200 which mayinclude index information. For example, in some embodiments, storagemanager component 195 may track or monitor the amount of available spaceand other similar characteristics of data associated with primary volume190.

In some embodiments, as further described herein, storage managercomponent 195 may also issue alerts or take other actions when theinformation associated with primary volume 190 satisfies certaincriteria, such as alert criteria, criteria defined in a storage policy,a threshold, such as a user defined threshold or default threshold, orother criteria. In some embodiments, action may be taken or alerts sentif a characteristic relating to a storage manager, media agent,database, storage device, storage library, client or other componentmatches the criteria. Such criteria may relate to resourcecharacteristics, including, for example, network bandwidth utilization,resource capacity used or available, time period between storageoperations, scheduling conflicts, processor loads, or othercharacteristics relating to performing a storage operation.

Second storage operation cell 173 may be directed to another type ofstorage operation, such as HSM storage operations. For example, thesecond storage operation cell 173 may perform backups, data migrations,snapshots, or other types of HSM-related operations known in the art.For example, in some embodiments, data is migrated from faster and moreexpensive storage such as magnetic disk storage (i.e., primary storage)to less expensive storage such as tape storage (i.e., secondarystorage).

Thus, the second storage operation cell 173 includes client component186 in communication with the primary volume 191 storing data. In someembodiments, client component 186 and primary volume 191 are the samephysical devices as the client component 185 and primary volume 190 inthe first storage operation cell 170. Similarly, in some embodiments,the storage manager component 196 and its associated database (notshown, and which may include index information) in the second storageoperation cell 173 are the same physical devices as the storage managercomponent and index in the first storage operation cell 170. Storagemanager component 196, however, typically also contains HSM modules orother logic associated with the second storage operation cell 173directed to performing HSM storage operations on primary volume 191data.

Storage operation cell 173 therefore may also contain media agent 206and a secondary storage media 211 configured for performing HSM-relatedoperations on primary copy data stored on primary volume 191. Forexample, storage manager 196 may migrate primary copy data from primaryvolume 191 to secondary volume 211 using media agent 206. Storagemanager 196 may also track and store information associated with primarycopy migration and other similar HSM-related operations in its storagemanager database (not shown). For example, in some embodiments, storagemanager component 196 may direct HSM storage operations on primary copydata according to a storage policy associated with the primary copy 191and stored in an index. In some embodiments, storage manager 196 mayalso track where primary copy information is stored, for example insecondary storage 211. Storage manager 196, media agents 206, 208, 210,and 212, and secondary storage media 213 and 217 may also be configuredto perform such HSM storage operations with respect to primary copy dataassociated with clients 236, 238, 240, 242, and 244. Further, storageoperation cell 175 may similarly be configured for performing such HSMstorage operations with respect to primary copy data associated withclients 246, 248, and 250.

Storage operation cell 180 may include a master storage manager 215 anda master storage manager 220. In some embodiments (not shown),additional storage operation cells may be located hierarchically inbetween storage operation cell 180 and storage operation cells 170, 173,and 175. In some embodiments, additional storage operation cellshierarchically superior to operation cell 180 may also be present in thehierarchy of storage operation cells.

In some embodiments, storage operation cells 170, 173, and 175 may beconnected by communications link 197, which may be any suitablecommunications link such as a fiber channel or SCSI connection thatallows storage operation cells 170, 173, and 175 to communicate directlywith one another (i.e., without necessarily directly or indirectlyinvolving storage operation cell 180). Such communication between cells170 and 173 may be accomplished, for example, by storage manager 195communicating with storage manager 196 via link 197. This allows storageoperation cells 170 and 173 to share information such as status oroperational characteristics with one another such as availability onprimary or secondary storage, trigger events based on such information,etc. This allows for the direct transfer of stored data to and from thestorage operation cells (via link 197) without the need to communicatewith or pass data through master storage manager 215.

Storage operation cell 180 is also generally directed to performingcertain types of storage operations, such as integration of SRM and HSMdata from other storage operation cells, such as storage operation cells170, 173, and 175. In some embodiments, fourth storage operation cell180 also performs other types of storage operations and might also bedirected to HSM, SRM, or other types of storage operations for otherreasons such as convenience, hardware allocation, media managers,administrative or organization reasons, etc.

FIG. 4 is a schematic depiction of migration of various components orresources, such as clients, media agents, primary storage, secondarystorage databases, storage managers or other components from one storageoperation cell to another storage operation cell in accordance withcertain aspects of the present invention. As shown, this may include,for example, movement of certain resources from storage operation cell173 to storage operation cells 170 and 175 in the illustrativeconfiguration of FIG. 3. As described herein, migration of storageoperation cell resources may be used to optimize, consolidate,reorganize or otherwise reconfigure or transfer storage resources orcomponents within the system. As mentioned above, migration of resourcestypically involves logical reassignment of certain resources, but may,in certain situations, also involve physical movement.

Logical migration may include associating or assigning a resource toanother storage operation cell. This may involve logically reassigningthe resource to another storage operation cell by transferring themanagement information associated with that resource, as well asreassigning that resource itself to another storage operation cell. Forexample, a media agent may be transferred from one storage operationcell to another by transferring management information regarding themedia agent to the new storage operation cell and logically reassigningthe media associated with that agent to the new storage operation cell(without any actual physical movement). Thus, for example, storagemanager database entries may be copied and repopulated with new links,path, and address information, etc. as necessary to complete thetransfer and bring the transferred resources to a fully operationalstate within the new storage operation cell.

Migration of resources may be initiated by a storage manager, such as amaster storage manager, or other component, or by a user, according touser selections or preferences. For example, a user may identify certainresources within a storage operation cell that are operating beyond aspecified capacity or that are underutilized, and mark those resourcesas possible migration candidates (e.g., placed in a migration pool).

Alternatively, a storage manager, or other system component, may monitorstorage operation cells to determine whether storage operation cellresources are performing adequately or as desired, e.g., within athreshold or according to an operating criteria or characteristic, orotherwise detect that storage operation cell configurations areperforming below an optimal, required or desired level. Based onacquired information, resources may be migrated to provide loadbalancing, promote efficient use of resources (e.g., storage media),improve data throughput, or provide other resource optimization.Resources may also be migrated and/or combined for convenience oradministrative or organizational purposes (e.g., merge or close physicaloffices etc.).

In general, migrating a resource from a source storage operation cell toa target storage operation cell may involve logically associating themigrated resource with target storage operation cell (withoutnecessarily requiring actual physical movement). Information relating tothe logical association(s) of migrated components may be stored in anindex, metabase, or database, for example database 220 (e.g., a storagemanager database). Such logical associations may include pointers to anew storage operation cell, a network pathway, or other indication thata storage operation cell resource has migrated. For example, if a clientis moved from one storage operation cell to another, information fromthe source storage operation cell including information from a storagemanager and/or associated media agent(s) may also be migrated to theappropriate resources in the target storage operation cell; and apointer or other referential element may be placed in the source storageoperation cell to indicate the client's new location.

In some embodiments, substantially all of this information may be movedto the target storage operation cell from the source storage operationcell. For example, some or all information such as managementinformation in an index or metabase may be migrated along with theresource so the resource may be reconstituted and resume normaloperation in the target storage operation cell. This may include bothdata and media (e.g., media agents and their associated medialibraries).

However, in certain circumstances, some or all of this managementinformation may remain in the source storage operation cell. Forexample, a client may be moved from one storage operation cell toanother, but one or more storage devices that store data from thatclient may remain in the source storage operation cell. In this case,this data from that client may remain in the source storage operationcell along with associated management information.

In the case where data needs to be retrieved from the source storageoperation cell, the target storage operation cell may include a pointeror other reference to the appropriate location in the source storageoperation cell. In other embodiments, however, associated storagedevices and/or the data itself associated with the migrated client maybe migrated to the target storage operation cell, in which casesubstantially all management information may be also be migrated to thetarget storage operation cell. In yet other embodiments, managementinformation may remain in both storage operation cells with both sets ofinformation updated as changes occur. In yet other embodiments, data maybe present in one storage operation cell with the client in a differentstorage operation cell, with pointers and references at the storagemanagers of each cell for interoperation.

In one example, a storage policy may indicate that a storage operation,such as a secondary copy operation is to be performed on a daily basisfor a particular client using a particular media agent and storagedevice. However, if the client is removed from the network and no longerproducing data on which a storage operation is performed, the mediaagent and storage device associated with storage operations for thatclient may be migrated to another storage operation cell for other uses(even if the media agent is still associated with clients in theoriginal cell). Alternatively, if a client is reassigned within anetwork from one department or domain to another, the associated mediaagent and storage device may be reassigned or referenced to a storageoperation cell that is already associated with the other department ordomain, or even reassigned or referred to some other storage operationcell.

In another example, clients may be added to an existing storageoperation cell, and may generate data that exceeds media capacity of oneor more storage devices in a storage operation cell. In this example, astorage device from another storage operation cell may be identified tomigrate to the storage operation cell to provide additional mediacapacity, or alternatively, one or more of the clients may be migratedto another storage operation cell that includes sufficient storagecapacity to handle the load. In other embodiments, a client may be splitor partitioned between two or more storage operation cells. In thiscase, the data agent associated with the client may be registered foroperation in those storage operation cells as well, with some data in afirst storage operation cell and some data in other storage operationcells.

Other situations causing bottlenecks, underutilization, orover-utilization of storage management resources may also arise in whichit may be useful or desirable to migrate or reassign one or moreresources to improve storage operation performance.

In FIG. 4, the outline of the migrated devices or components and thechanged data/information communication paths in the source storageoperation cell 173 are depicted by dashed lines to indicate theirmigration, transfer, or removal from the source storage operation cell,and the migrated devices or components (identified by identicalreference numerals) are depicted by solid lines in the target storageoperation cell (i.e., cells 170 and 175) to indicate their migration tothat target storage operation cell.

More specifically, as shown, client 240 and data from storage manager196 may be migrated from source storage operation cell 173 to targetstorage operation cell 175 (including storage manager 260) such thatmedia agent 216 (or other media agent(s) of storage operation cell 175)is operative in managing data movement (e.g., for a storage operation,such as a backup, recovery, or migration) between primary storage volumeof client 240 and secondary storage media 223. Transferring client 240may occur and be desirable, for example, in the event that client 240 ismoved from one enterprise department to another department, or in theevent that media agent 210 previously associated with client 240 becomesoverburdened, or is otherwise unavailable to perform specifiedfunctions. It will be understood that client 240 may be reassignedwithout any physical movement of client 240, but rather by movement ofmanagement information and logical reassignment within the system. Suchan operation may sometimes be referred to herein as “migration” of oneor more resources from one storage operation cell to another.

Continuing with the embodiment illustrated in FIG. 4, client 244together with its associated media agent 212 may be migrated from sourcestorage operation cell 173 to storage operation cell 175, with mediaagent 212 subsequently being associated with secondary storage media 225to manage data movement between client 244 (i.e., its primary storagevolume) and secondary storage media 225. Client 244 its associatedstorage manager data and possibly its associated media agent 212, andmedia library and any other necessary or useful routing or managementdata may be migrated or transferred, for example, in the event that astorage volume 219 reaches a capacity that exceeds a threshold orcriteria, or in another scenario in which it is desirable to migrateclient 244 and media agent 212.

Continuing with the illustrative embodiment, client 186 (including itsprimary storage volume 191 and storage manager data) may be migratedfrom source storage operation cell 173 to storage operation cell 175such that media agent 214 (or other media agent in cell 175) isoperative in managing data movement (e.g., for a storage operation, suchas a backup, recovery, or migration) between primary storage volume 191and secondary storage media 221. Client 186 and its associated storagemanager data may be migrated or transferred, for example, because mediaagent 214 is underutilized in storage operations and has capacity whichmeets a threshold or criteria, or media agent 206 is overburdened andexceeds a bandwidth threshold or criteria, or satisfies other criteriain which it may be desirable to transfer client 186.

Moreover, media agent 206 may be migrated from source storage operationcell 173 to storage operation cell 170 and associated with secondarystorage media 227 to conduct data between client 185 or other clients(not shown) and secondary storage media 227, providing for a storageoperation, or a backup of data on primary storage volume 190.Alternatively or additionally, media agent 206 may remain associatedwith secondary storage media 211 to direct a storage operation of aprimary storage volume 190 thereto, and, in such cases, secondarystorage media 211 may also be migrated to storage operation cell 170,and its association with media agent 208 may be removed. Such migrationmay occur, for example, in the event that media agent 206 has acharacteristic such as bandwidth, meeting a threshold or criteria toperform storage operations relating to client 185 and media agent 208has a characteristic, such as bandwidth, to take over control of storagemedia 211. In other situations, it may be desirable to move client 185to another storage operation cell, without media agent 206, and reassignclient 185 to a new media agent in the new storage operation cell. Thismay involve migrating some or all of management from agent 206 and anyassociated storage manager information associated with the client asfurther described herein.

It will be understood, the migration of the resources as illustrated inFIG. 4 may be based on the following illustrative situation prior tomigration. With regard to media agent 206, in addition to handlingstorage operation data movement between primary storage volume 191,media agent 206 may also be operative in performing storage operationsfor primary storage volume 190 of client 185 via link 197. The workloadof handling data movement in a storage operation for both of theseprimary storage volumes may overload media agent 206 and causecongestion. Likewise, clients 238, 240, and 242 may over utilize mediaagent 210, thus causing congestion and delay, for example, in networkpathways to media agent 210, and also rapidly consume the storagecapacity of storage media 217 and 219. Further, performing a storageoperation of client 244 via media agent 212 increases the storagecapacity consumption rate of media 219.

In this example, load balancing techniques and reallocation within a thestorage operation cell alone may not adequately mitigate these problems,or may not represent an optimal solution compared to possibly migratingone or more resources to another storage operation cell (with or withoutmigrating resources within the storage operation cell).

Load redistribution may be addressed according to the resource migrationdescribed above in connection with FIG. 4. Thus, client 186 (withprimary storage volume 191) and client 244 and their associated data maybe migrated to storage operation cell 175, assigning each of theseclients to a respective media agent and associated secondary storagethat will not be overloaded or over-utilized by the addition of theseclients. With the migration of client 186, media agent 206 may now beavailable to manage and control storage operations for client 185, andthus it may be migrated to storage operation cell 170. Because of theaddition of media agent 206 and secondary storage media 227 (and/or, inalternative implementations, secondary storage media 211) into storageoperation cell 170, HSM components may be added to storage manager 195to allow for HSM operations, including collection and reporting of HSMdata.

To further reduce the consumption rate of storage media 219, client 244may be migrated to storage operation cell 175. Because media agent 212is associated with client 244, it may be migrated to storage operationcell 175 with client 244 without requiring migration of other clientsconnected thereto, and without requiring any migration of databases(e.g., indexes) that media agent 212 would maintain for such otherclients. In alternative implementations, a media agent already locatedin storage operation cell 175 may be assigned to client 244, or a newmedia agent for client 244 may be installed and configured in storageoperation cell 175. In either case, storage preferences, storagepolicies or other preferences provided for a storage operation cell,media agent or other component of a storage operation cell may beretained by the resource or component after migration to a targetstorage operation cell, or alternatively the preference may beoverwritten with new preferences associated with the target storageoperation cell.

Accordingly, it will be understood that the ability to migrate ortransfer resources across storage operation cells provides for greaterflexibility in solving load balancing problems, providing robustfailover solutions, and providing means by which systems efficiency maybe monitored and improved providing, for improved performance,reliability, and resource utilization. Such load balancing or failoverproblems can be monitored and remedied on a system-wide or global basis,as well as on an individual storage component or resource level.

Referring now to FIG. 5, a flow chart illustrating some of the steps formigrating one or more resources from a source storage operation cell toa target storage operation cell is provided in accordance with anembodiment of the present invention. As noted above, resource migrationmay be implemented or initiated according to a process including any ofthe following: (1) one or more steps executed automatically in responseto user input; (2) one or more steps executed automatically without userinput in accordance with certain system operating policies orguidelines; or (3) one or more manually executed steps with certainsteps occurring automatically. It will be understood that the processdescribed with reference to FIG. 5 is merely illustrative, and that eachoperation or series of operations may be implemented automatically,manually, or a combination thereof.

For example, a user may initiate the resource migration process byinteracting via a GUI (or other user interface) with a network resourcemanagement system that provides information and functions for storageoperation cell resource allocation (step 505). Resource management maybe implemented as one or more software modules or applications that maybe integrated with, or be a component of, for example, the HSM and/orSRM software. At least some of the information monitored or generated bythe HSM and/or SRM components, or other storage manager components maybe used by the network resource management system for allocating ormigrating resources. For instance, corrective actions recommended by theHSM component may be used by the resource management system indetermining whether migration of a resource to another storage operationcell should be recommended.

In another example, a storage manager or other system component mayreceive information relating to, or may periodically query storageoperation cell or system resources, or consult an index or database orstorage policy regarding storage operation cell or system resources todetermine current and future scheduled storage operation cell andresource utilization, and initiate the resource migration process, (step505). Such periodic resource query may be the system checks or testsdescribed in commonly owned U.S. patent application Ser. No. 11/269,513,titled SYSTEM AND METHOD FOR MONITORING A STORAGE NETWORK, filed Nov. 7,2005. Thus, a storage manager or other system component may determinecurrent and/or predicted resource needs or requirements for a storageoperation cell, for example, based on a storage policy or otherinformation and allocate storage operation cell resources accordingly.

The network resource management system, such as a storage managercomponent, or other component, may identify one or more resources to bemoved or migrated from a given storage operation cell based, forexample, on system operating conditions, user preference, and/orcombination of the two, which may include additional forecasting orprovisioning analysis (step 507). For instance, the network resourcemanagement system may provide information (e.g., component storagecapacity, rate of storage media consumption, scheduled storageoperations, network data pathway load metrics, processor loads, andother performance metrics) concerning resources (e.g., clients, mediaagents, primary or secondary storage media, data paths, or othercomponents) within each of the storage operation cells. Alternatively,in the case where migration of resources is accomplished automatically,such information may be obtained by a storage manager, master storagemanager or other system component to calculate performance metrics anddetermine and/or identify storage operation cell resources formigration. This may be performed on a substantially global basis, and amaster storage manager component or other management component mayanalyze operations of the storage operation cells in a network and makerecommendations and/or suggestions for resource migration based on thisinformation (e.g., based on certain system optimization or data routing,management, and movement algorithms, etc.).

The resource management system or storage manager may also generatealerts identifying operating conditions that require attention in thestorage operation cells, such as a pathway or processor bottleneck, oneor more storage operations that exceed a threshold relating to systemperformance which may be remedied by resource reallocation (e.g., asfurther described in FIG. 6). The network resource management system orstorage manager may also recommend the migration or reallocation ofspecific resources, and may also identify one or more potential targetlocations (within or without the source storage operation cell) for themigration or reallocation. This may occur, for example, in recognitionof impending hardware, software, or other system failures, such thenetwork may be may correct problems, prior to occurrence to maintain orpreserve system operation and data protection criteria and/or preventservice degradation (e.g., a “self-healing” capability in view ofidentified and impending component failures).

The alerts and/or recommendations for migrating one or more resourcesgenerated by various storage managers (or other storage components) maybe based on analysis of certain information (e.g., storage capacity,rate of storage media consumption, data path load metrics, and otherperformance metrics or other system diagnostics) concerning resources(e.g., clients, media agents, secondary storage media, data paths)within its storage operation cell including existing configurations ofthe storage operation cell, schedule policy or other preferences inadvance of a storage operation.

Additionally or alternatively, the network resource management systemmay itself analyze information received or acquired directly orindirectly from the storage managers. For instance, such informationfrom each storage operation cell may be stored on a globally accessibledatabase. In a hierarchical configuration, such as shown in FIG. 3, thisinformation from each storage operation cell may be communicated to, andstored in the database of, the master storage manager. The networkresource management system may then access and analyze this informationto identify situations for which resource migration, or other correctiveor preventative actions, may be recommended. Thus, the network resourcemigration system may facilitate identifying resources to be migrated. Itwill be understood, however, that in some situations the decision tomigrate a resource across storage operation cells may be identified viathe resource management system based on specific performance scenarios,configurations, resource characteristics or other basis.

Based on the information, alerts, or recommendations generated by thesystem, or based on other considerations, the user may select via theGUI (or other interface means) a resource located in a source storageoperation cell for assignment or re-assignment to a specified targetstorage operation cell (step 509). In some embodiments, the selectionmay be made automatically by the system based on system operatingconditions and metrics as determined by performance algorithms or otheranalysis software. In such an embodiment, the resource automaticallyselected may be identified as having appropriate utilization formigration to a target storage operation cell. Possible resources thatmay be selected include a client, a media agent, storage media,transmission resources and a datapath (i.e., a combination of mediaagent and storage media and transmission path used by the storage policyto perform a data protection operation), or other system component.

In some embodiments, prior to assigning a resource to a target storageoperation cell, a storage manager or other system component may directthat storage operations be quiesced or temporarily ceased and thatcertain confirmatory test(s) be performed to verify that the anticipatedor intended storage operations can be performed in the new proposedstorage operation cell configurations before allowing the new storageoperation cell configurations to be brought online. In some embodiments,when the storage operations are buffered or quiesced, a storage manageror other system component may observe the source and target storageoperation cells and any components which may be associated or affectedby the migrated resource to determine whether the migration is operatingas anticipated.

In other embodiments, a storage manager or other system component maysimulate the new resource configurations of the source and targetstorage operation cell in advance of the migration, to determine whetherthe configuration is desirable and likely to achieve the desired goal.The simulation may take into account existing information relating toeach storage operation cell resource and storage operation scheduling orother job related information which may be obtained from a storagepolicy. This may include simulation of one or more proposedconfigurations by analysis or optimization software, and choosing thebest one based on specified criteria through the use of, for example, aselection algorithm (or user input).

The system may then provide for moving management information associatedwith each selected resource (and stored in the source storage operationcell storage manager) from the source storage manager to the targetstorage manager (step 511). This may include any management informationassociated with the migrated resource such as index informationassociated with a media management component, metadata associated withthe resource or other information. The communication of this informationmay occur, for example, directly between storage managers of the sourceand target storage operation cells, or indirectly through one or moreintermediary storage managers, which may include a master storagemanager.

In some embodiments, a copy of all or part of the management informationmay be physically transferred to the target storage manager. By way ofexample, when migrating a client, the management information that istransferred to the target storage operation cell includes media records,archive records, application records, job history, source and targetnetwork pathway information and some or all storage policy criteria forthat client.

The transfer of this information may be accomplished by firstidentifying or obtaining the information in the source storage operationcell, and copying the information, or merging the information in, to thetarget storage operation cell. This may include the techniques describedin co-pending application entitled “Systems and Methods of UnifiedReconstruction in Storage Systems”, which claims priority from U.S.Provisional Application 60/752,202, filed Dec. 19, 2005, both of whichare hereby incorporated by reference in their entirety.

In some embodiments, to identify or obtain this information, the systemmay create a folder or other destination into which the system storesthe information. Then, the target storage operation cell may beconfirmed and any other designations that may be required by the systemdepending on the resource being transferred and/or the target storageoperation cell. For example, in migrating a client, it may be requiredto designate a library and a media agent in the target storage operationcell corresponding to a given storage policy. The system may merge theinformation stored in the folder into the storage manager of the targetstorage operation cell.

In some embodiments, the management information may be stored to astorage manager database or index. For example, metadata or indexinformation relating to the resource being migrated may be copied to acorresponding media management component in the target storage operationcell. This allows the media management component such as a storagemanager, media agent or other component in the target storage operationcell to properly to manage and interoperate with the transferredresource in the same or similar manner as in the source storageoperation cell. This typically includes associating the managementinformation with the new storage operation cell and associatedcomponents and resources, etc., to ensure interoperation. As mentionedabove, the migrated resource (e.g., the client or other component) maybe configured to use the storage manager of the target storage operationcell. This resource configuration may occur, for example, prior totransferring the management information in step 511.

In the event that the management information is incomplete, or there isan inconsistency or other failure between management information storedby the resource and that stored by the storage manager (step 513), thena restore operation may be performed, such as an incremental restoreoperation. The restore operation may be implemented in an incrementalorder, accessing the data and metadata stored on one or more storagemedia (e.g., from one or more jukeboxes) to restore the data andmetadata in reverse time order, thus restoring the metadata in thetarget storage operation cell (e.g., the metadata stored in the storagemanager as well as in clients and/or media agents). As such, informationrelating to the incremental order may be obtained and restored, (step515). However, other restore methods may be used, if desired, such as byuse of copying data blocks from other locations, checking pointers toresources or other components, according to the method described inco-pending commonly owned U.S. Provisional Patent Application Ser. No.60/752,202, SYSTEMS AND METHODS FOR UNIFIED RECONSTRUCTION OF DATA IN ASTORAGE NETWORK filed Dec. 19, 2005, and its corresponding utilityapplication 25963/012.

Once the management information is received by the target storageoperation cell and the transferred resource has been configured withrespect to its new location (i.e., other resources configured tointeroperate with the transferred resource as contemplated) the targetstorage manager, the transferred resource may be assigned to one or morestorage devices (step 517) by specifying, for example, particular mountpaths, network pathways, and/or storage media. The assignment mayinclude logically associating one or more pathways and the migratedresource and may be based on past or predicted performance of resources.Such associations may be stored in an index and may be provided usingpointers or other methods.

The system may perform certain diagnostics to verify the resourcemigration configuration is as intended and check for errors (step 519).In some embodiments, diagnostics or storage operation cell configurationsimulation may be performed on the source and/or target storageoperation cell in advance of performing a storage operations on thesource or target storage operation cells. This may be done to confirmthat the resource migration was successful or to identify problems thatmay be present in the new configurations (and if identified, take stepsto address or repair such problems).

Simulations of new configurations may include calculating knowncharacteristics of a migrated resource and other storage operation cellresources together with metrics relating to storage operations toprovide a confidence factor that may indicate whether the newconfiguration will provide the intended benefit. Based on one or moreconfiguration confidence factors, a highest scoring configuration may beselected to be implemented. In the event that a resource is migrated,and the storage manager or other system component determines that yetanother configuration would provide enhanced storage performance, thestorage manager may analyze this information and initiate the migrationprocess in an iterative fashion.

In general, information, including optimization levels, confidencefactors, performance improvement, and other information relating tostorage operation cell migration is stored and tracked to provideongoing metrics that may be used to optimize future storage operationcell configurations.

FIG. 6 is a flow chart illustrating some of the steps involved inproviding dynamic resource allocation in accordance with certainembodiments of the present invention. As will be understood, thisdynamic allocation procedure may be used for various dynamicreallocation of resources including, but not limited to, load balancingand/or failover configurations, predictive or corrective analysisreconfiguration, optimizations etc.

As shown, the operation of the resources in each storage operation cellmay be monitored (step 705) by the system. For instance, the monitoredinformation may include storage capacity, rate of storage mediaconsumption, data path load metrics, and other performance metrics asfurther described herein. Such system monitoring may be automatic andestablished, for example, according to a user preference or defaulttemplate defined by a system administrator setting forth performance orconfiguration requirements or preferences. The monitoring may beperformed as described in commonly owned U.S. patent application Ser.No. 11/269,513, SYSTEM AND METHOD FOR MONITORING A STORAGE NETWORK,filed Nov. 7, 2005.

The system may compare or otherwise evaluate the operation of theresource with respect to one or more thresholds (step 710). Thethresholds may be established in a storage policy, user preference orother preference as mentioned above. The comparison or evaluation may bebased on individual comparisons of monitored metrics, indicia, and otherparameters with respective individual thresholds. For example, athreshold for a secondary storage device may be a 70% utilization rate,above which additional media may be required to perform storageoperations, or as another example, data path usage exceeding aparticular capacity (e.g., 80%).

Alternatively, or additionally, comparisons or evaluations may be madebased on a metric that is a function (e.g., a use specified weighted sumor average) of monitored quantities and/or other metrics, and which maycharacterize the overall operating level of an individual resource orthe storage operation cell as a whole (e.g., risk of data loss). Forexample, media usage and predicated growth metrics may be used todetermine whether a particular media library can be continued to beused. If, for example. A media library is within a certain time periodof being fully utilized, based on media usage and growth metrics (e.g.,3 days), the system may generate a warning, request new media for thelibrary, and determine alternate destinations, should the library becomesubstantially fully utilized (e.g., 95% capacity).

In the event that the operation of any resource, or overall storageoperation cell operation is within the preset thresholds, (step 715),storage operation cell monitoring may continue on a constant or periodicbasis. At some point, the monitoring may indicate that a particularresource exceeds a certain threshold or criteria. The system mayidentify this resource as a potential resource to reallocate and/oridentify other resources to allocate to the storage operation cell toremedy the condition causing the threshold to be exceeded. The potentialresources for reallocation may be identified by determining whetherresources in other storage operation cells for which the operatingthresholds are not exceeded (e.g., underutilized resources) areavailable for migration (step 720). A list of such resources may bemaintained by one or management components within the system and used toidentify and consider which resources may be best be used to solve aparticular problem. Moreover, a list of under utilized resources may bekept track of as a measure of the “reconfigurability” of system to adaptto changing conditions.

For example, if a client, secondary storage device, media agent or otherresource in another storage operation cell is operating within a pre-setthreshold or in accordance with a criteria, such resources may beavailable to migrate to the target storage operation cell which includesresources which exceed the preset thresholds. Resource information maybe obtained by locating a pool of resources, querying a list ofresources, comparing resources with job scheduling, storage policies orother information, pinging or otherwise communicating with resources todetermine what resources may be underutilized or over utilized. Cellinformation may also be obtained by consulting a storage manager,storage manager index, media agent, or other system component.

Potential resource reallocations may also include resources physicallylocated within or logically associated with the storage operation cellof the resource to be transferred. In the case that more than onepotential reallocation is identified, the system prioritizes them basedon considerations such as how well the resource needs can be satisfied,load balancing, data loss risk, as well as other conditions which may bedefined in advance by the user.

The system may also determine whether other conditions or constraintsexist that prevent allocation or transfer of the resource, (e.g.,physical location, lack of communications paths, incompatible with oneanother, impending or actual hardware failure, etc.) (step 725). If theresource cannot be allocated to any of the identified potential targetlocations, then the system may remove the resource from considerationfor migration (step 730). Alternatively, the system may alert a user ofthe condition and allow the user to review the status of the resources.Also, in advance of an automatic migration, the system may send an alertto an administrator for authorization to perform the resource transfer.The system may then return to step 720 to locate other availableresources for allocation. and/or for permission to proceed. In otherembodiments, the system may have been preauthorized and the migrationmay occur substantially automatically.

In some embodiments, the system may determine (at step 720) thepotential optimization improvement obtained by transferring theresource. For example, a storage manager or other system component maysimulate the operation of a target storage operation cell including acomponent or resource migrated from a located storage operation cell andthe operation of the located storage operation cell without the migratedresource. Such simulation may include determining whether the migrationwill enhance or otherwise improve performance of storage operations andwhether the new target and source storage operation cell configurationswill continue to function appropriately. This may involve evaluation ofcertain system performance metrics such as growth rate of target, futurescheduled jobs, routing, pathway and temporal constraints, etc. andsimulate operation of proposed new configuration(s) for a certain periodof time in the future to confirm the desired benefits, are in fact,likely to be achieved. Moreover, multiple such proposed configurationsmay be evaluated, with the m system (or user) selecting the mostdesirable option.

In the event that the resource can be transferred (step 730), the systemmay automatically initiate the transfer, (step 735), (e.g., identifyingand copying data relating to the resource, merging the data andreconfiguring the resource, updating the index with new logicalassociations of the new configuration, etc.). Data relating to theresource transfer may be stored in a storage manager or media agentindex or database. Following the resource transfer, the system maycontinue to periodically monitor the operation and resource utilizationof the storage operation cells.

In accordance with a further embodiment of the present invention,resource reallocation may be based on predictive and other analyses, aswill be further understood in view of the illustrative operational flowshown in FIG. 7. As shown, at step 805, the operation of the resourcesin each storage operation cell may be monitored by the system. Themonitored information may include monitoring trends or patterns such asthose relating to storage capacity, rate of storage media consumption,data path load metrics, system congestion, data load with respect totime, or any other performance metrics of interest to a system operator.This information may be communicated among storage operation cells,and/or to a master storage manager or other globally accessible location(step 810).

The monitored information may be used to perform predictive analysistype operations on a resource, storage operation cell, and/or networkwide basis (step 815). Predictions may relate to individual monitoredparameters or metrics, or functions thereof, and may involve loadanalysis, level of data protection, risk criteria of a storagecharacteristics, cost of storing data, data availability, or otherindicia or metrics concerning the resources or the state of storageoperations in one or more storage operation cells. In some embodiments,the predictive analysis may be performed according to user, department,project, or other identifier. Such predictions may be performed in orderto identify and correct resource shortages and/or system inefficienciesprior to their occurrence.

In operation, a user may interact with the system via the GUI (or otherinterface) to specify and perform a desired analysis, and access and/orgenerate reports for trends concerning resource allocation. In someembodiments, the user may specify alternative configurations of theresources and perform a predictive analysis for such alternativeconfigurations based on the monitored information for the resources inthe current configuration. In some embodiments, the predictive analysisis performed automatically, for example according to user predefinedanalysis; a storage policy; or other preference. Simulations of storageoperation cell operations and/or configurations may also be tracked andused to optimize future storage operation cell operations and/orconfigurations.

Trends and patterns obtained in the predictive analysis may be usedtogether with other information such as storage policy information todetermine future resource use required to perform storage operationsaccording to the storage policy and to prevent resource exhaustion,overuse and system inefficiencies. For example, the system may predict,based on growth trends within a certain storage operation cell, the needfor additional media agents, storage media, transmission paths or otherresources (e.g., within a certain time frame). Such resources may bemigrated from one storage operation cell to another prior to a lack oroveruse of such resources becomes problematic in a given cell. Forexample, the system may predict an impending shortage of storage mediawithin a certain storage operation cell. The system may consider variousoptions for solving this problem and choose the optimal solution(including adding additional media to existing storage devices). Thismay involve migrating certain storage devices into the storage operationcell to help alleviate the predicted impending shortage of storagemedia.

Another example involves migrating additional media agents or data pathsto a storage operation cell to help prevent bottlenecks or congestionconditions that may be predicted based on current growth or operationaltrends. For example, the system may predict media agent utilization willexceed a predefined utilization threshold based on client or datagrowth. In addition to considering other options, the system may locateand migrate (or prepare to migrate) additional media agents andreconfigure associated transmission paths to help alleviate this problemprior to its occurrence or before reaching certain level of severity.

Another scenario involves the case where device or component failurerates are trending above acceptable limits (e.g., for media agent hosts,storage manager hosts, storage devices, storage media, etc.). The systemmay recognize this emerging problem and reassign certain resources suchas clients and/or media agents to other transmission paths to minimizethis problem, or the potential for this problem. The system may alsokeep track of certain system configuration and operational informationregarding system components such as hardware type, expected lifespan (invarious configurations), current deployment time etc. This informationmay be evaluated to determine the likelihood of failure and systemreconfiguration and/or device replacement recommendations or actions maybe made based on this information. This may be accomplished using thesystems and methods described in co-pending U.S. patent application Ser.No. 11/120,619, HIERARCHICAL SYSTEMS AND METHODS FOR PROVIDING STORAGE AUNIFIED VIEW OF STORAGE INFORMATION, filed May 2, 2005, and U.S. PatentApplication Ser. No. 60/752,197 titled SYSTEMS AND METHODS FORHIERARCHICAL CLIENT GROUP MANAGEMENT, filed Dec. 19, 2005, both of whichare incorporated herein by reference.

Moreover, the system may monitor client growth in order to predict apoint in time when it is desirable to split a storage operation cellinto two or more storage operation cells or, alternatively when it maybe desirable to combine multiple storage operation cells into onestorage operation cell. For example, a storage operation cell may besplit into multiple storage operation cells once the number of clientsand/or media agent reach a certain threshold point. This point may bedetermined by certain utilization thresholds set as part of userpreferences or system requirements, or may be based on bandwidth ofcomputational abilities of such resources (e.g., predicted inability toor certain resources to handle operations with a certain time frame,etc.).

In operation, the system may recognize the need to split storageoperation cells and create additional storage operation cells andmigrate the appropriate resources into those storage operation cells asdescribed above. Similarly, one or storage operation cells may becombined into a single storage operation cell once resource utilizationfalls below a certain threshold level. Such consolidation may also occurin the event of physical relocation or reallocation of certain resourcesto promote the efficient grouping or configuration of resources withinthe system.

Thus, the system may determine the future resources and future storageoperation cell configurations required to perform storage operations andcompare the resource requirement and storage operation cellconfiguration to a current resource allocation (step 820), such asexisting components associated in one or more storage operation cells.Based on the comparison, resources may be reallocated, (e.g., migratedfrom a source storage operation cell to a target storage operationcell), either automatically by the system or in response to userspecified reallocation, and/or a need for further modifications and/orenhancements to the storage network may be established (step 825). Afterthe resources have been reallocated, the system may monitor trends instorage characteristics so that the trend or pattern predictiveinformation may be dynamically updated accordingly and used to configurestorage operation cell configurations on an ongoing basis a describedherein (step 830). Accordingly, the system according to one embodimentis able to continuously maintain an optimized and/or fail-safe storagesystem even as over time the system changes in its usage, resources,capacity, setup etc.

Moreover, as mentioned above, resources or data may be migrated from onestorage operation cell to another (existing or new) storage operationcell on a temporary or permanent basis. For example, in disasterrecovery or other failure scenario, if one or more storage operationcells are experiencing failures, or have lost or are in jeopardy oflosing power or will otherwise become unable to continue normaloperation because of loss of resources or communications links (e.g.,impending natural disaster), the system may create a storage operationcell and migrate resources to that storage operation cell on a temporarybasis such that the resources may be returned or otherwise reallocatedat some point in the future once the problem has been corrected.

As another example, assume a certain storage operation cell has lostpower and its resources are currently operating with a backup powersupply such as a UPS (uninterruptible power supply). The system may beable to operate for only a limited period of time based on the UPSpower. The system may recognize the problem, and create a temporarystorage operation cell and migrate clients and data from the storageoperation cell that has lost power to the temporary storage operationcell. Such a migration may be scheduled to be substantially completebefore the UPS runs out of power. This allows data and other systemoperations to continue to operate substantially uninterrupted untilpower can be restored to the original storage operation cell. At thispoint, data and storage operation cell resources may be returned to theoriginal storage operation cell. In some embodiments, certain resourcesmay be assigned to such temporary storage operation cells to act as afailover storage operation cell in such situations. This may be done toensure data protection and system robustness even if certain storageoperation cells are rendered inoperable by unforeseen or uncontrollablecircumstances.

Systems and modules described herein may comprise software, firmware,hardware, or any combination(s) of software, firmware, or hardwaresuitable for the purposes described herein. Software and other modulesmay reside on servers, workstations, personal computers, computerizedtablets, PDAs, and other devices suitable for the purposes describedherein. Software and other modules may be accessible via local memory,via a network, via a browser or other application in an ASP context, orvia other means suitable for the purposes described herein. Datastructures described herein may comprise computer files, variables,programming arrays, programming structures, or any electronicinformation storage schemes or methods, or any combinations thereof,suitable for the purposes described herein. User interface elementsdescribed herein may comprise elements from graphical user interfaces,command line interfaces, and other interfaces suitable for the purposesdescribed herein. Screenshots presented and described herein can bedisplayed differently as know in the art to input, access, change,manipulate, modify, alter, and work with information. Except to theextent necessary or inherent in the processes themselves, no particularorder to steps or stages of methods or processes described in thisdisclosure, including the Figures, is implied. In many cases the orderof process steps may be varied, and various illustrative steps may becombined, altered, or omitted, without changing the purpose, effect orimport of the methods described.

Accordingly, while the invention has been described and illustrated inconnection with preferred embodiments, many variations and modificationsas will be evident to those skilled in this art may be made withoutdeparting from the scope of the invention, and the invention is thus notto be limited to the precise details of methodology or construction setforth above as such variations and modification are intended to beincluded within the scope of the invention.

What is claimed is:
 1. A method of automatically transferring primarydata from a first storage operation cell to an offsite second storageoperation cell, the method comprising: copying at least a first portionof primary data from a first storage operation cell to one or moresecondary storage devices to create secondary data of the primary data,the first storage operation cell comprising at least a first storageresource and at least a first operational characteristic associated witha first threshold; identifying a second storage operation cellcomprising a second storage resource having a second operationalcharacteristic that does not exceed a second threshold, the secondstorage resource located at an offsite location that is different than alocation of the first storage resource; automatically transferring atleast a portion of the primary data stored on the first storage resourceto the second storage resource associated with the second storageoperation cell if the first operational characteristic is near orexceeds the first threshold, and automatically transferring a copy of atleast a portion of management information associated with the firststorage operation cell to the second storage operation cell; anddetermining that the copy of the portion of the management informationassociated with the second storage operation cell is incomplete andperforming a restore operation from at least a portion of the secondarydata stored in the one or more secondary storage devices to restore thecopy of the portion of the management information stored in associationwith the second storage operation cell.
 2. The method of claim 1 whereinthe management information is at least metadata associated with at leastone of the group consisting of: resources, media records, archiverecords, application records, job history, source and target networkpathway information, and storage policy criteria.
 3. The method of claim1 further comprising verifying the copy of the portion of the managementinformation transferred from the first storage operation cell.
 4. Themethod of claim 1 wherein management information remains in the firststorage operation cell after the copy of the portion of the managementinformation is transferred to the second storage operation cell.
 5. Themethod of claim 4 further comprising updating management information inboth the first storage operation cell and the second storage operationcell when changes occur.
 6. The method of claim 1 wherein transferringthe copy of the portion of the management information migrates the copyof the portion of the management information from the first storageoperation cell to the second storage operation cell.
 7. The method ofclaim 1 wherein first operational characteristic is representative of adistribution of storage operations within the first storage operationcell.
 8. The method of claim 1 wherein the first operationalcharacteristic is representative of storage media consumption.
 9. Themethod of claim 1 wherein the first operational characteristic isrepresentative of overall system performance.
 10. The method of claim 1wherein transferring at least a portion of the first storage resource tothe second storage operation cell occurs after transfer of the copy ofthe portion of the management information.
 11. A storage system thatautomatically transfers primary data from a first storage operation cellto a second storage operation cell, the storage system comprising: afirst storage operation cell comprising computer hardware, the firststorage operation cell copies at least a first portion of the primarydata from the first storage operation cell to one or more secondarystorage devices to create secondary data of the primary data, the firststorage operation cell comprising at least a first storage resource andat least a first operational characteristic associated with a firstthreshold; a second storage operation cell comprising computer hardware,the second storage operation cell comprising a second storage resourcehaving a second operational characteristic that does not exceed a secondthreshold, the second storage resource located at an offsite locationthat is different than a location of the first storage resource; astorage manager comprising computer hardware, the storage manager copiesa portion of portion of the primary data stored first storage resourceto the second storage resource associated with the second storageoperation cell if the first operational characteristic is near orexceeds the first threshold, and the storage manager automaticallytransfers a copy of a portion of the management information associatedwith the first storage operation cell to the second storage operationcell to keep the first operational characteristic with the firstthreshold; and wherein the second storage operation cell determines thatthe copy of the portion of the management information associated withthe second storage operation cell is incomplete and performs a restoreoperation from at least a portion of the secondary data stored in theone or more secondary storage devices to restore the copy of a portionof the management information stored in association with the secondstorage operation cell.
 12. The storage system of claim 11 wherein themanagement information is at least one of metadata associated with thegroup of first resources, media records, archive records, applicationrecords, job history, source and target network pathway information, andstorage policy criteria.
 13. The storage system of claim 11 wherein thesecond storage operation cell verifies the copy of the portion of themanagement information transferred from the first storage operationcell.
 14. The storage system of claim 11 wherein management informationremains in the first storage operation cell after the copy of theportion of the management information is transferred to the secondstorage operation cell.
 15. The storage system of claim 14 whereinmanagement information in both the first storage operation cell and thesecond storage information cell are updated when changes occur.
 16. Thestorage system of claim 11 wherein the transfer of the copy of theportion of the management information migrates the copy of the portionof the management information from the first storage operation cell tothe second storage operation cell.
 17. The storage system of claim 11wherein the first operational characteristic is representative of adistribution of storage operations within the first storage operationcell.
 18. The storage system of claim 11 wherein the first operationalcharacteristic is representative of storage media consumption.
 19. Thestorage system of claim 11 wherein the first operational characteristicis representative of overall system performance.
 20. The storage systemof claim 11 wherein the transfer of the at least a portion of theprimary data from the first storage resource to the second storageoperation cell occurs after the transfer of the copy of the portion ofthe management information.